Packet-switched networks, such as networks based on the TCP/IP protocol suite, can be utilized to distribute a rich array of digital content to a variety of different client applications. The most popular applications on the Internet today are browsing applications for searching the World Wide Web, e.g. Netscape Navigator or Microsoft Internet Explorer, which utilize the Hyper Text Transfer Protocol (HTTP) to retrieve documents written in the Hypertext Markup Language (HTML) along with embedded content. See, e.g., R. Fielding et al., “Hypertext Transfer Protocol—HTTP/1.1,” IETF RFC 2616, Network Working Group, 1999, which is incorporated by reference herein. HTML documents, as well as other resources on the Internet such as embedded multimedia content, are addressed by Uniform Resource Locators (URLs), e.g. “http://www.xyz.com/dir/document.html” which identifies an HTML document, “document.html” on server “www.xyz.com” in directory “dir” which may be accessed using the HTTP protocol. See Berners-Lee, T., et al., “Uniform Resource Locators,” IETF RFC 1738, Network Working Group, 1994, which is incorporated by reference herein. Servers/hosts are identified by domain names, e.g. “www.xyz.com”, which are part of a loosely hierarchical naming scheme which are mapped into network IP addresses using the Domain Name Service (DNS). See P. Mockapetris, “Domain names—concepts and facilities,” RFC 1034, November 1987; P. Mockapetris, “Domain names—implementation and specification,” RFC 1035, November 1987; which are incorporated herein by reference. DNS is in essence a distributed database of multiple name servers that maintain and answer queries on mapping between domain names and addresses. Name servers belong to a hierarchy wherein DNS queries are resolved by contacting other name servers and following a delegation/referral chain to an authoritative name server for the queried host. For example, before a client can issue a request for a resource identified in a particular URL, a DNS query must be issued to translate the host name into an IP address that is utilized to establish the connection to the server with the desired resource.
It is often advantageous when distributing digital content across a packet-switched network to divide the duty of answering content requests among a plurality of geographically dispersed servers. For example, extremely popular Web sites on the Internet often provide links to “mirror” sites which replicate the content at a number of locations across the globe, some closer to the particular client requesting the content. A more recent alternative to mirroring has been the use of what are referred to in the art as “content distribution” services. Content distribution services provided by companies such as Akamai Technologies, Digital Island, and Adero utilize architectures for their content provider customers which dynamically redirect content requests to a cache advantageously situated closer to the client issuing the request (such architectures are referred to herein generically as “content distribution networks” or “CDNs” for short). These companies either co-locate caches within Internet Service Providers or deploy them within their own separate networks.
Content distribution offerings differ in the ways they divide the functions and control over request processing between the customer and the CDN platform. There are primarily two alternative content distribution architectures: (1) systems which rewrite URLs in documents to point to the cached content; and (2) systems which rely on outsourcing of DNS lookups. In the former, accesses to a customer's “root” page (such as “www.xyz.com”) can be served by the customer's own Web site, but embedded content such as images, music, or movies can be served using a modified URL (referred to herein by the inventors as a “convoluted URL”). Special scripts may be executed by the customer that modify embedded URLs to use the domain name of the content distribution network. In the latter type of content distribution architecture, the content distribution service provides an authoritative DNS name server(s) for part or all of the customer's Web site. For example, “www.xyz.com” may be served by the “xyz” company's own server but “images.xyz.com” might be resolved by the CDN.
Although there are several CDNs in the market today with more likely to appear in the future, there are disadvantages to relying on any one content distribution service. It is often difficult to provision the proper amount of resources for the CDN service while deploying the service fast enough to be able to reach all possible clients with the desired quality, especially where the content provider expects high-load either continuously or in bursts due to a particularly popular event. It is desirable to have a CDN service that is not constrained by its current capacity and footprint and that can build up capacity and footprint as the service grows.